Digital Negative
Digital Negative (DNG) is an open raw image format owned by Adobe used for digital photography. It was launched on September 27, 2004.[1] The launch was accompanied by the first version of the DNG specification,[2] plus various products including a free of charge DNG Converter utility. All Adobe photo manipulation software (such as Adobe Photoshop and Adobe Lightroom) released since the launch supports DNG.[3]
DNG is based on the TIFF/EP standard format, and mandates significant use of metadata. Exploitation of the file format is royalty free; Adobe has published a license allowing anyone to exploit DNG,[4] and has also stated that there are no known intellectual property encumbrances or license requirements for DNG.[5] Adobe stated that if there was a consensus that DNG should be controlled by a standards body, they were open to the idea.[6] Adobe has submitted DNG to ISO for incorporation into their revision of TIFF/EP.[7]
Rationale for DNG
Given the existence of other raw image formats, Adobe's creation of DNG as a competing format implies that DNG is unusual and satisfies objectives that other raw image formats do not. These objectives and the associated characteristics of DNG, as well as assessments of whether these objectives are met, are described below. Increasingly, professional archivists and conservationists, working for respectable organizations, variously suggest or recommend DNG for archival purposes.[8][9][10][11][12][13][14][15][16]
Although it is possible to verify that DNG has a given characteristic, it is difficult to demonstrate that no other relevant format offers the same. Therefore, anyone who is concerned about a particular objective/characteristic can check the status of DNG here, but should also check the status of alternatives elsewhere. (There is further reading at the articles on raw image formats, dcraw, and reverse engineering).
Objectives
These objectives are repeatedly emphasized in Adobe documents:[1][6][17][18]
- Digital image preservation (sometimes known as "archiving"): to be suitable for the purpose of preserving digital images as an authentic resource for future generations.[19] Assessment: The US Library of Congress states that DNG is a recommended alternative to other raw image formats: "Less desirable file formats: RAW; Suggested alternatives: DNG".[20] The Digital Photography Best Practices and Workflow (dpBestflow) project, funded by the United States Library of Congress and run by the American Society of Media Photographers (ASMP), singles out DNG, and states "DNG files have proven to be significantly more useful than the proprietary raw files in our workflow".[21]
- Easy and/or comprehensive exploitation by software developers: to enable software to be developed without the need for reverse engineering; and to avoid the need for frequent software upgrades and re-releases to cater for new cameras. Assessment: Software could support raw formats without DNG, by using reverse engineering and/or dcraw; DNG makes it easier, and many software products can handle, via DNG, images from cameras that they have no specific knowledge of.[22] An unresolved restriction is that any edit/development settings stored in the DNG file by a software product are unlikely to be recognized by a product from a different company. (This problem is not specific to DNG).
- In-camera use by camera manufacturers: to be suitable for many camera manufacturers to use as a native or optional raw image format in many cameras. Assessment: About 12 camera manufacturers have used DNG in-camera. About 38 camera models have used DNG.[23] Raw image formats for more than 230 camera models can be converted to DNG.[24]
- Multi-vendor interoperability: to be suitable for workflows where different hardware and software components share raw image files and/or transmit and receive them.
Characteristics
All of the above objectives are facilitated or enabled by most of these characteristics:[25]
- Freely-available specification:[26] this can be downloaded from the Adobe website without negotiation or needing justification.[2]
- Format based on open specifications and/or standards: DNG is compatible with TIFF/EP, and various open formats and/or standards are used, including Exif metadata, XMP metadata, IPTC metadata, CIE XYZ coordinates, ICC profiles, and JPEG.[2][27]
- Self-contained file format: a DNG file contains the data (raw image data and metadata) needed to render an image without needing additional knowledge of the characteristics of the camera.[2][22]
- Version control scheme: it has a version scheme built into it that allows the DNG specification, DNG writers, and DNG readers, to evolve at their own paces.[28][29]
- Freely-available source-code-based software development kit (SDK): there are 3 aspects - there is an SDK; it is source-code-based (as can be verified by examination); and it can be downloaded from the Adobe website without needing justification.[30]
- Documented to have no known intellectual property encumbrances or license requirements: there is both a "Digital Negative (DNG) Specification Patent License" which says that anyone can exploit DNG,[4] and a statement that there are no known intellectual property encumbrances or license requirements for DNG.[5]
Technical summary
A DNG file always contains data for one main image, plus metadata, and optionally contains at least one JPEG preview.[2] It normally has the extension "dng" or "DNG".
DNG conforms to TIFF/EP and is structured according to TIFF. DNG supports various formats of metadata, (including Exif metadata, XMP metadata, IPTC metadata), and specifies a set of mandated metadata.[28]
DNG is both a raw image format and a format that supports "non-raw", or partly processed, images.[2] The latter (non-raw) format is known as "Linear DNG".[31] Linear DNG is still scene-referred and can still benefit from many of the operations typically performed by a raw converter, such as white balance, the application of a camera color profile, HDR compositing, etc. All images that can be supported as raw images can also be supported as Linear DNG. Images from the Foveon X3 sensor or similar, hence especially Sigma cameras, can only be supported as Linear DNG.
DNG can contain raw image data from sensors with various configurations of color filter array (CFA). These include: conventional Bayer filters, using 3 colors and rectangular pixels; 4 color CFAs, for example the RGBE filter used in the Sony DSC-F828; rectangular (non-square) pixels, for example as used in the Nikon D1X; and offset sensors (for example with octagonal pixels) such as Super CCD sensors of various types, as used in various Fujifilm cameras. (Or combinations of these if necessary). DNG specifies metadata describing these individual parameters; this is one significant extension to TIFF/EP.
When used in a CinemaDNG movie clip, each frame is encoded using the above DNG image format. The clip's image stream can then be stored in one of two formats: either as video essence using frame-based wrapping in an MXF file, or as a sequence of DNG files in a specified file directory.
Timeline
This provides a mixture of the dates of significant events (such as "the first X") and various counts of usage at the anniversaries of the launch (each 27 September).
Counts of products and companies that use DNG in some way are provided primarily for illustration. They are approximate, and include products that are no longer sold. The purpose is mainly to demonstrate that such products and companies exist, and to show trends. Convertible raw image formats (camera models whose raw images can be converted to DNG) only include official support by Adobe DNG converters; not unofficial support by Adobe products (sometimes reaching about 30), nor support by other DNG converters.[24]
- 2003, late: Adobe started work on the DNG specification.[6]
- 2004, early: Adobe started talking to other companies about DNG.[6]
- Launch, 2004, 27 September: Adobe launched DNG. Specification version 1.0.0.0 published.[1] Convertible raw image formats: 60+
- 2005, February: Specification version 1.1.0.0 published.
- 2005, June: First digital back to write DNG, the Leica DMR (Digital Modul R) back for the R8 and R9.
- 2005, July: First camera manufacturer to provide a DNG converter for its own raw file formats - Hasselblad's FlexColor.
- 2005, July: First genuine digital SLR camera to write DNG, Hasselblad H2D.
- 1st anniversary, 2005, 27 September: Camera manufacturers: 4. Camera models: 7. Software products: 70+. Convertible raw image formats: 70+.
- 2005, October: First compact camera to write DNG, Ricoh GR Digital.
- 2006, July: First monochrome digital back to write DNG, MegaVision E Series MonoChrome.
- 2006, September: First rangefinder camera to write DNG, Leica M8.
- 2006, September: First camera to offer the user a choice of proprietary raw or DNG, Pentax K10D.
- 2nd anniversary, 2006, 27 September: Camera manufacturers: 8. Camera models: 9. Software products: 120+. Convertible raw image formats: 110+.
- 2007, May: First raw converter & photo-editor whose first raw-handling release only supported DNG, Pixelmator.
- 2007, July: First underwater camera to write DNG, Sea&Sea DX-1G. (Based on Ricoh Caplio GX100).
- 3rd anniversary, 2007, 27 September: Camera manufacturers: 10. Camera models: 13. Software products: 170+. Convertible raw image formats: 160+.
- 2007, October: First digital scan back system and first 360-degree panorama system to write DNG, Seitz 6x17 Digital and Seitz Roundshot D3 with D3 digital scan back.[32][33]
- 2008, February: First software on a mobile phone to write DNG, Tea Vui Huang's "DNG Phone Camera" for Nokia.
- 2008, April: Adobe announces CinemaDNG initiative, using DNG as the basis for the individual raw images of a movie.
- 2008, May: Specification version 1.2.0.0 published.
- 2008, September: First movie camera to use DNG as a raw image format, Ikonoskop A-cam dII.[34]
- 4th anniversary, 2008, 27 September: Camera manufacturers: 13. Camera models: 29. Software products: 200+. Convertible raw image formats: 180+.
- 2008, September: First DNG converter running on Linux (among several other things), digiKam.
- 2009, spring/summer: First digiscope with built-in camera to write DNG, Zeiss Photoscope 85 T* FL.[35]
- 2009, June: Specification version 1.3.0.0 published.
- 5th anniversary, 2009, 27 September: Camera manufacturers: 14. Camera models: 38. Software products: 220+. Convertible raw image formats: 230+.
- 2009, November: First "interchangeable unit" camera to write DNG, Ricoh GXR.[36]
- 2010, February: First 3D movie camera to write DNG, Ikonoskop A-cam3D.[37]
- 2010, March: First medium format camera to offer the user a choice of proprietary raw or DNG, Pentax 645D.
- 6th anniversary, 2010, 27 September: Camera manufacturers: 14. Camera models: 47. Software products: 240+. Convertible raw image formats: 290+.
During the first 5 years when about 38 camera models were launched that wrote DNG, Adobe software added support for about 21 Canon models, about 20 Nikon models, and about 22 Olympus models. Camera models that write DNG are almost certainly launched at a much higher rate than camera models that support any other individual raw image format, including NEF, CR2, and ORF.
Reception
The reaction to DNG has been mixed.[24] A few camera manufacturers stated their intention to use DNG at launch. The first supported DNG about 9 months after launch. Several more niche and minority camera manufacturers added support after this. The largest camera manufacturers have apparently never indicated an intention to use DNG.
A few software products supported DNG within 5 months of launch. Many more added support after this. A few software products only support DNG from cameras that write DNG, and/or from cameras that they support via their native raw image formats.
OpenRAW was an advocacy and lobby group with the motto "Digital Image Preservation Through Open Documentation". They became opposed to DNG.
DNG conversion
"DNG conversion" refers to the process of generating a DNG file from a non-DNG image. (This is in contrast to "raw conversion", which typically refers to reading and processing a raw file, which might be a DNG file, and generating some other type of output from it). DNG conversion is one of the sources of DNG files, the other being direct output from cameras and digital backs.
Several software products are able to do DNG conversion. The original such product is Adobe DNG Converter or DNG Converter, a freely available stand-alone utility from Adobe.[17] Other Adobe products such as the ACR plugin to Photoshop or Lightroom can also generate DNG files from other image files.
Most DNG converters are supplied by companies other than Adobe. For example:
- The software that Pentax supply with all their dSLR cameras can convert PEF raw image files from Pentax cameras into DNG files.
- Flexcolor and Phocus from Hasselblad can convert 3FR raw image files from Hasselblad cameras and digital backs into DNG files.
- Capture One from Phase One is a raw converter that can process not only raw image files from Phase One digital backs, but also raw image files from many other cameras too. Capture One can save images from many of those cameras to DNG.
- KDE Image Plugin Interface is an API, that can save the images it is processing to DNG. It can be used standalone or with any image processing applications of the KDE desktop environment under Linux and Windows.
- A number of DNG converters have been developed by "amateurs" to enable raw images from their favored camera or digital back to be processed in a large range of raw converters. These include cases where cameras have been hacked to output raw images that have then been converted to DNG.[38]
The process of DNG conversion involves extracting raw image data from the source file and assembling it according to the DNG specification into the required TIFF format. This optionally involves compressing it. Metadata as defined in the DNG specification is also put into that TIFF assembly. Some of this metadata is based on the characteristics of the camera, and especially of its sensor. Other metadata may be image-dependent or camera-setting dependent. So a DNG converter must have knowledge of the camera model concerned, and be able to process the source raw image file including key metadata. Optionally a JPEG preview is obtained and added. Finally, all of this is written as a DNG file.
DNG conversion typically leaves the original raw image file intact. For safety, many photographers retain the original raw image file on one medium while using the DNG file on another, enabling them to recover from a range of hardware, software, and human, failures and errors. For example, it has been reported in user forums that some versions of the Adobe DNG Converter don't preserve all the raw data from raw images from some camera models.[39][40]
Summary of products that support DNG in some way
This section summarizes other more comprehensive lists.[41][42]
Adobe products
All raw image file handling products from Adobe now support DNG.[3] Adobe DNG Converter was published by Adobe Systems on September 27, 2004. It converts different camera raw format files into the Digital Negative (DNG) standard. It also supports lossless data compression when converting. The program is free of charge, but not open source. It can be downloaded at Adobe's site (Win and Mac).
Digital cameras and related software
Use by camera manufacturers varies; there are about 15 camera manufacturers that use DNG, including a few that specialize in movie cameras:[23]
- Niche camera manufacturers typically use DNG in new cameras. (Niche cameras include a digiscope, panorama cameras, and at least one movie camera). The article on raw image formats illustrates the complicated relationship between new raw image formats and third-party software developers. Using DNG provides immediate support for these cameras by a large range of software products.
- Some camera manufacturers with low market share but conventional cameras use DNG in new cameras. Camera manufacturers that don't supply their own software for processing raw images typically, but not always, use DNG.
- One company, Pentax, offers users the option of whether to use Pentax's own raw image format, or DNG. For example, the digital SLR Camera Pentax K-x does offer the ability to save PEF or DNG or even DNG+ which saves two files, a DNG and a separate JPEG file at the same time.
- If a camera uses DNG, and that camera manufacturer supplies software, it will support DNG. It may support DNG only from their own cameras, or support it more generically.
- Except Pentax, no camera manufacturer with a large market share of dSLRs, such as Canon, Nikon, Sony, Panasonic, Olympus, Fuji, Sigma and some others, use DNG in their cameras. (However, raw images files from those cameras can be converted to DNG). If a camera manufacturer's cameras don't use DNG, their software is unlikely to support DNG unless that software is also sold independently of the cameras.
Here are examples of digital cameras that supported DNG.[23]
- Casio supports DNG in their Exilim PRO EX-F1 and Exilim EX-FH25.
- Leica's Digital Modul R for the Leica R8 or Leica R9 and the Leica M8 or Leica M9 natively support the DNG format.
- MegaVision E Series Monochrome back.
- Panoscan MK-3 digital panoramic camera.
- Pentax supports DNG in their K10D, K20D, K200D, K2000, K-7, K-x, K-r and K-5 DSLR cameras.
- Ricoh supports DNG in the Ricoh Digital GR, considered a professional compact, and the Ricoh Caplio GX.
- Ricoh GXR is the latest, new and solely system of mirrorless interchangeable lens camera unit use also DNG.[43]
- Samsung supports DNG in their Pro815 "prosumer" camera and GX-10 and GX-20 DSLR cameras.
- Sea&Sea DX‐1G underwater camera.
- Seitz Roundshot D3 digital back, used in cameras such as the 6×17.
- Silicon Imaging Silicon Imaging Digital Cinema SI-1920HDVR.
- Sinar now uses DNG as the raw file standard for their eMotion series of digital backs.
Some of the Canon cameras can shoot as DNG using additional free software named CHDK.[38]
Third party software
Support by software suppliers varies; there are of the order of 200 software products that use DNG.[41][44]
The majority of raw handling software products support DNG. Most provide generic support, while a few support it only if it output directly from a camera. The type of support varies considerably. There appear to be very few third party software products that process raw images but don't support DNG. This may reflect the difficulty of discovering all of those that don't.[45]
Versions of the specification
All versions of the specification remain valid, in the sense that DNG files conforming to old versions should still be read and processed by DNG readers capable of processing later versions. DNG has a version scheme built into it that allows the DNG specification, DNG writers, and DNG readers, to evolve at their own paces.[29] Each version of the specification describes its compatibility with previous versions.[2]
- 1.0.0.0, published September 2004
- This version accompanied the launch of DNG and related products. It was a rare, possibly unique, example of a raw image format specification published by its owner. It was adequate for representing typical images, but it had a few errors and deficiencies (specifically the lack of support of "masked pixels" and an inadvertent deviation from the JPEG specification) that required it soon to be replaced by the next version.
- 1.1.0.0, published February 2005
- This version corrected the flaws in the first version. It has proved capable of representing raw images for a large variety of cameras (both when written in-camera or via conversion from other raw image formats) for a few years, and it is the version still typically written in-camera.
- 1.2.0.0, published May 2008
- This version was based on experience and feedback from other companies about DNG since its launch. It introduced many new features, especially several new options for color specification under the general heading of "Camera Profiles". These are mainly of value to software products wanting their own flavor of color handling. This version permits administrative control of Camera Profiles, including calibration signatures and copyright information.
- 1.3.0.0, published June 2009
- This version added various improvements, but the major change was to introduce "Opcodes". A Opcode is an algorithm to be applied to some or all of the image data, described in the specification, and (optionally) implemented in the product that reads and processes the DNG file. The DNG file itself holds lists of Opcodes to be executed, together with the parameters to be applied on execution. In effect, the DNG file can contain lists of "function calls" to be executed at various stages in the raw conversion process. For example, the WarpRectilinear Opcode "applies a warp to an image and can be used to correct geometric distortion and lateral (transverse) chromatic aberration for rectilinear lenses". This is an example of an algorithm that cannot be applied to the raw image data itself before it is placed into the DNG file, because it should be executed after demosaicing. There are 13 Opcodes described in this version, and each Opcode is accompanied by a specification version so that more can be added in future.
- CinemaDNG, published September 2009
- CinemaDNG uses DNG for each frame of a movie clip. There are additional tags specifically for movies: TimeCodes and FrameRate.[46] It is not clear whether these tags will be added to a later version of the DNG specification, or will remain separately described only in the CinemaDNG specification.
Standardization
DNG is not (yet) a standard format, but is based on several open formats and/or standards and is currently being used by ISO in its revision of TIFF/EP. Here is a summary of the timeline:
- 2001: The ISO standard raw image format, ISO 12234-2, better known as TIFF/EP, was ratified and published. (TIFF/EP also supports "non-raw", or "processed", images). TIFF/EP provided a basis for the raw image formats of a number of cameras, but they typically added their own proprietary data. Some cameras have sensors that cannot be described by the current version of TIFF/EP.
- 2004, September: Adobe launched DNG based on TIFF/EP. (The DNG specification states "DNG ... is compatible with the TIFF-EP standard").[2] DNG is an extension of TIFF/EP with considerably more specified metadata. In effect, DNG was TIFF/EP brought up-to-date and made fit for purpose. DNG also exploits various other open formats and/or standards, including Exif metadata, XMP metadata, IPTC metadata, CIE XYZ coordinates, ICC profiles, and JPEG.[2][27] Although DNG supports more sensor configurations than TIFF/EP, (for example, cameras from Fujifilm using Super CCD sensors), it still doesn't support all sensor types as raw images, especially those using the Foveon X3 sensor or similar, hence especially Sigma cameras.
- 2006: TIFF/EP began its 5-year revision cycle.
- 2006 to 2007 sometime: Adobe offered the DNG specification to ISO to be part of ISO's revised TIFF/EP standard.[7][47]
- 2008, September & October: Minutes of ISO/TC 130/WG2 — Prepress Data Exchange, 37th Meeting: "WG 18 is revising the two-part standard (ISO 12234), which addresses digital camera removable memory. The revision of ... Part 2 will add DNG into TIFF/EP." A progress report from ISO about the revision of TIFF/EP stated that the revision "... currently includes two "interoperability-profiles," "IP 1" for processed image data, using ".TIF" extension, and "IP 2" for "raw" image data, ".DNG" extension".[48]
- 2009, September: Minutes of ISO/TC 130/WG2 — Prepress Data Exchange, 39th Meeting: the revision of TIFF/EP "is comprehensive to support many different use cases, including backward compatibility with current TIFF readers and support of Adobe DNG.... Profile 2 (proposed extension .dng, if Adobe is in agreement) is intended for camera raw images, including un-demosaiced images.... This format will be similar to DNG 1.3, which serves as the starting point for development."[49]
- Future?: If ISO's revision of TIFF/EP continues as it has been doing, (this is not guaranteed), there will be an ISO Standard with one of its interoperability profiles based on DNG version 1.3 and used with the file extension ".DNG" or ".dng".
See also
References
|
|
|
| Desktop software |
|
|
|
| Readers and players |
|
|
| Server software |
|
|
| Technology |
|
|
| Web Services |
|
|
| Board of directors |
|
|
| Acquisitions |
|
|
|
|
|